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Checklists  and  Criteria  for  Evaluating  the  Cost  and 
Schedule  Estimating  Capabilities  of  Software 
Organizations 


Abstract.  This  report  provides  criteria  and  checklists  for  evaluating  the  capability 
of  an  organization's  software  estimating  process  and  the  infrastructure  that  sup¬ 
ports  it.  It  also  supplies  guidelines  for  good  estimating  practice.  The  checklists 
and  guidelines  can  be  used  to  elicit  information  for  process  assessments  and  to 
motivate  and  guide  organizations  in  process  improvement  efforts. 


1.  Introduction 

This  report  has  four  components: 

•  A  list  of  reasons  organizations  have  for  estimating  software  size,  effort,  cost,  and 
schedule. 

•  A  checklist  of  requisites  for  reliable  estimating  processes.  This  checklist  can  be  used 
to  focus  and  guide  assessments  of  process  maturity.  It  identifies  six  requisites  of 
reliable  estimating  processes  and  provides  examples  of  evidence  to  look  for  as 
indicators  of  process  maturity. 

•  A  checklist  for  successful  estimating  environments.  This  checklist  can  be  used  by 
enterprise  managers  to  identify  issues  to  address  when  seeking  to  establish  and 
sustain  a  corporate  software  estimating  capability.  It  can  be  used  also  as  a  guideline 
for  evaluating  the  commitment  and  support  an  organization  provides  for  its  estimating 
process. 

•  A  summary  of  important  elements  of  good  estimating  practice. 

These  materials  provide  criteria  and  guidelines  to  help  organizations  assess  the  capability  of 
software  estimating  processes  and  the  infrastructures  that  support  them.  They  arm  you  with 
questions  to  ask  and  examples  of  evidence  to  look  for  when  assessing  the  capability  of  an 
estimating  process  and  the  organization's  commitment  to  make  the  process  work. 

Although  we  prepared  these  materials  to  help  people  assess  the  processes  and  practices 
used  to  estimate  software  costs  and  schedules,  almost  everything  in  this  report  applies 
equally  to  hardware  and  integrated  systems  projects  as  well.  If  you  have  responsibilities  for 
developing  hardware  or  integrated  systems,  you  may  find  that  altering  the  word  'software' 
wherever  it  appears  will  make  the  materials  useful  beyond  just  the  software  functions  of  your 
organization. 

The  checklists  and  tables  in  this  report  were  prepared  as  part  of  the  SEI's  Software  Cost 
Estimating  Improvement  Initiative.  Please  let  us  know  if  you  find  them  helpful,  or  if  you  have 
suggestions  for  improving  their  usefulness  for  your  organization.  For  a  closely  related 
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checklist  that  provides  guidance  for  evaluating  individual  estimates,  please  see  A  Manager's 
Checklist  for  Validating  Software  Cost  and  Schedule  Estimates  [Park  95]. 


2.  Why  Do  We  Estimate  Software  Size,  Effort,  Cost,  and 
Schedule? 


There  are  many  reasons  to  estimate  the  size,  effort,  cost,  and  schedule  of  software  products 
and  projects.  Table  1  on  the  next  page  lists  the  ones  that  we  identified  during  our  work  on 
the  Software  Cost  Estimating  Improvement  Initiative  [Park  94],  We  present  this  list 

•  To  give  visibility  to  the  variety  of  reasons  why  reliable  estimating  processes  are 
important. 

•  To  help  you  identify  opportunities  for  getting  your  money's  worth  from  the  estimates 
you  prepare  or  receive. 

As  Colonel  Russell  Logan  of  the  Air  Force  Pentagon  Communications  Agency  observed 
when  reviewing  this  work: 

Cost  estimating  should  be  a  corporate  process — an  essential  one  at  that — 
and  not  something  to  be  singled  out  for  or  subject  to  budgetary  axing  just  to 
save  costs.  It  is  either  an  essential  part  of  your  business  or  it  is  not.  If  not, 
any  effort  expended  on  estimating  is  meaningless. 

In  a  downsizing  epoch  (such  as  many  organizations  face  today,  both  in 
government  and  business),  estimating  becomes  a  tool  in  your  engineering 
and  management  belt  and  not  a  piece  to  be  sold  off. 

While  cost  and  schedule  estimating  has  not  been  well  done  historically,  it 
remains  nonetheless  an  essential  element  in  the  tools  required  by  all 
software  professionals.  Effective  estimating  is  necessary  if  a  business  is  to 
survive  and  thrive.  It  is  as  essential  as  knowing  the  target  language  and 
environment,  and  it  must  become  part  of  the  bottom  line  for  any  corporate 
structure. 
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Table  1:  Reasons  for  Estimating  Software  Size, 
Effort,  Cost,  and  Schedule 


To  scope  proposed  software  tasks 

To  explore  the  affordability  of  a  system 

To  explore  alternative  system  concepts 

To  explore  alternative  design  concepts 

To  explore  alternative  proposals  for 
enhancements  and  upgrades 

To  identify  key  design  elements 

To  identify  key  process  parameters 

To  identify  key  assumptions 

To  identify  key  cost  drivers,  so  they 
can  be  properly  managed 

To  identify  uncertainties  and  quantify 
risks 

To  identify  and  manage  major  risk 
items 

To  set  priorities 

To  help  plan  the  necessary  steps  for 
completing  a  project 

To  identify  tasks  and  their  relationships 

To  assess  schedule  feasibility 

To  identify  and  evaluate  cost  and 
schedule  tradeoffs 

To  plan  for  staffing  profiles  and 
manpower  buildups  that  meet  project 
needs 

To  allocate  and  schedule  resources 
To  establish  budgets 
To  get  funding 

To  assess  an  organization's  ability  to 
perform  within  targeted  costs 


To  avoid  underestimating  the  magnitude 
and  complexities  of  software  projects 

To  evaluate  the  consequences  of 
internal  and  external  constraints 

To  establish  achievable  objectives 

To  establish  a  basis  for  quality  service 

To  establish  commitments 

To  bound  the  risk  of  commitments 

To  balance  levels  of  risk  against 
customer  needs 

To  provide  a  basis  for  successful  risk 
management 

To  prepare  successful  proposals 

To  provide  a  quantitative  basis  for 
presenting  proposed  costs  and 
schedules  to  customers 

To  inform  a  customer  of  the  potential 
cost  of  services  from  a  fee-for-service 
organization 

To  evaluate  proposals  from  competing 
bidders 

To  support  independent  reviews  of 
proposed  projects  (independent  cost 
estimates) 

To  serve  as  a  basis  for  negotiating  cost 
agreements 

To  establish  baselines  for  project 
tracking 

To  predict  life-cycle  costs 

To  predict  returns  on  investments 

To  provide  information  for  establishing 
business  strategies 


CM  U/SEI-95-S  R-005 


3 


3.  Requisites  and  indicators 


Reliable  cost  and  schedule  estimating  processes  share  a  number  of  important 
characteristics.  Table  2  lists  six  that  we  have  observed.  We  believe  these  to  be  requisites 
for  producing  estimates  that  organizations  can  trust. 


Table  2:  Six  Requisites  for  Reliable  Estimating  Processes 

1 .  A  corporate  memory  (historical  database) 

2.  Structured  processes  for  estimating  product  size  and  reuse 

3.  Mechanisms  for  extrapolating  from  demonstrated 
accomplishments  on  past  projects 

4.  Audit  trails  (Values  for  the  cost  model  parameters  used  to 
produce  each  estimate  are  recorded  and  explained.) 

5.  Integrity  in  dealing  with  dictated  costs  and  schedules 
(Imposed  answers  are  acceptable  only  when  legitimate 
design-to-cost  or  plan-to-cost  processes  are  followed.) 

6.  Data  collection  and  feedback  processes  that  foster  capturing 
and  correctly  interpreting  data  from  work  performed 


Trust,  of  course,  is  a  matter  of  degree.  Just  how  extensively  to  rely  on  an  organization's 
estimating  depends  on  how  thoroughly  that  organization  addresses  these  process  requisites. 

The  first  checklist  in  this  report  expands  upon  Table  2.  It  provides  you  with  elements  of 
evidence  to  look  for  when  assessing  the  capability  and  maturity  of  an  estimating  process.  It 
also  gives  you  a  structured  format  to  use  when  probing  for  evidence  and  recording  your 
observations. 

Although  Table  2  and  its  associated  checklist  are  useful  guides,  they  do  not  tell  you  all  that 
you  need  to  know  when  assessing  an  organization's  estimating  capability.  Reliable 
estimating  processes  don't  just  happen.  Developing  and  sustaining  any  process  requires 
organizational  commitment  and  action.  Table  3  supplements  Table  2  by  identifying  seven 
indicators  of  serious  and  sustained  commitment  to  reliable  estimating.  This  table  (and  the 
checklist  that  supports  it)  looks  at  the  commitment  and  support  an  organization  provides  for 
its  estimating  process,  rather  that  at  the  internal  structure  of  the  process  itself. 

Table  3  differs  from  Table  2  in  another  way  as  well.  It  is  a  list  of  indicators,  not  requisites. 
While  items  in  the  list  seem  to  be  good  things  to  do  (and  we  may  personally  feel  strongly 
about  them),  different  organizations — in  differing  situations — may  have  differing  approaches 
to  providing  sustainable  process  infrastructures  that  work  for  them.  Nevertheless,  the  extent 
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to  which  these  indicators  are  present  can  influence  our  assessment  of  an  organization's  long¬ 
term  commitment  to  treat  estimating  as  a  corporate  asset. 


Table  3:  Seven  Indicators  of  Estimating  Capability 

1 .  Management  acknowledges  its  responsibility  for  developing 
and  sustaining  an  estimating  capability. 

2.  The  estimating  function  is  supported  by  a  budget  and  funds. 

3.  Estimators  have  been  equipped  with  the  tools  and  training 
needed  for  reliable  estimating. 

4.  The  people  assigned  as  estimators  are  experienced  and 
capable. 

5.  Recognition  and  career  paths  exist  such  that  qualified  people 
want  to  serve  as  estimators. 

6.  Estimators  work  with  process  improvement  teams  to  quantify 
and  track  progress  in  software  process  improvement. 

7.  The  estimating  capability  of  the  organization  is  quantified, 
tracked,  and  evaluated. 


The  second  checklist  in  this  report  expands  upon  Table  3.  It  provides  a  guide  for  enterprise 
managers  to  use  when  planning  the  actions  they  will  take  to  make  their  estimating  process 
an  asset  they  can  rely  on.  Evaluators  can  also  use  the  checklist  to  probe  for  evidence  of  an 
organization's  support  for  its  estimating  process.  The  stronger  the  evidence,  the  stronger  the 
indication  that  the  organization  produces  (and  will  continue  to  produce)  estimates  that  can  be 
trusted. 

The  final  item  of  this  report  is  a  summary  of  some  important  elements  of  good  estimating 
practice.  This  summary  includes  examples  that  do  not  fit  neatly  in  a  checklist  of  process 
requisites,  but  that  are  worth  considering  when  implementing  an  estimating  process  or 
assessing  its  maturity.  We  have  organized  the  summary  in  list  format,  much  like  the 
checklists.  You  will  find  it  at  the  end  of  the  report,  immediately  following  the  two  checklists. 
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4.  Checklists  and  Elements  of  Good  Practice 


We  present  the  checklists  and  elements  of  good  practice  in  the  pages  that  follow.  You  may 
make,  use,  or  distribute  as  many  copies  as  you  wish,  so  long  as  you  reproduce  the  entire 
document  and  include  the  copyright  notice  in  each  case. 

We  have  made  no  attempt  to  prejudge  or  prioritize  the  importance  of  the  individual  items  of 
evidence  that  we  illustrate  in  the  checklists.  As  you  assign  priorities  or  weights  to  the 
evidence  you  find,  you  should  be  guided  by  the  size  and  type  of  the  organization,  its  products 
and  customers,  the  purposes  for  which  the  estimates  are  used,  and  the  nature  and 
combinations  of  the  evidence  you  observe.  In  almost  all  cases,  the  evidence  itself  (or  lack 
thereof)  will  be  your  best  guide  to  the  importance  to  place  on  what  you  find  in  any  particular 
assessment.  As  always,  the  total  picture  will  be  what  is  important.  The  checklists  simply 
help  you  probe  and  sort  through  the  details  that  give  substance  to  that  picture. 
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Requisites  for  Reliable  Estimating  Processes 
—  A  Maturity  Checklist  — 

This  checklist  is  designed  to  help  you  evaluate  the  maturity  of  an 
organization's  software  estimating  process.  It  can  be  used  to  elicit 
information  for  process  assessments  or  to  motivate  and  guide 
organizations  in  process  improvement  activities. 


Requisite  1.  A  corporate  memory  (historical  database) 

Evidence  of  Maturity 

The  organization  has  a  process  for  organizing  and  retaining 
information  on  completed  projects  (a  historical  database). 

The  historical  database  is  treated  as  an  integral  part  of  the 
estimating  process,  and  estimators  have  active  roles  in 
specifying  and  sustaining  the  information  it  contains. 

The  database  contains  a  useful  set  of  completed  projects. 

The  elements  included  in  (and  excluded  from)  effort,  cost, 
schedule,  size,  and  reuse  measures  are  clearly  identified. 

(For  examples,  see  the  SEI  checklists  for  effort,  schedule, 
and  size  measurement.) 

Schedule  milestones  (start  and  finish  dates)  are  described  in 
terms  of  criteria  for  initiation  or  completion. 

Effort  and  cost  data  clearly  indicate  which  parts  of  the  life  cycle 
and  which  activities  are  covered  by  the  different  categories  of 
hours  or  costs  recorded. 

Records  for  projects  indicate  whether  or  not  unpaid  overtime 
was  used. 

Unpaid  overtime,  if  used,  is  quantified,  so  that  recorded  data 
provide  a  valid  basis  for  estimating  future  effort. 

Cost  models  are  used  to  provide  a  consistent  framework 
(standard  terms  and  parameters)  for  recording  historical  data. 

Historical  data  have  been  examined  to  identify  inconsistencies, 
and  anomalies  have  been  corrected  or  explained. 

(This  is  perhaps  best  done  with  the  same  cost  models  that 
are  used  for  estimating.) 

Work-flow  schematics  are  used  to  describe  similarities  and 
differences  among  projects. 
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Information  on  completed  projects  includes 

•  The  life-cycle  model  used,  together  with  the  portion  covered  Q 
by  the  recorded  schedule  and  costs. 

•  The  original  size  estimate.  Q 

•  Changes  in  size  resulting  from  changes  in  requirements.  Q 

•  The  original  cost  and  schedule  estimate,  together  with  the  Q 

values  and  rationales  used  for  cost  model  parameters. 

•  Re-estimates  and  estimates-to-complete.  □ 

•  Reasons  for  re-estimates.  □ 


(Reasons  help  us  interpret  the  data.  Examples  include 
changes  in  requirements;  changes  in  priorities;  major 
surprises;  erroneous  estimates  of  size,  difficulty,  or  other 
parameters;  delays  due  to  resource  constraints;  and 
divergence  of  performance  from  plans.) 


•  Actual  costs  and  schedules.  □ 

•  Actual  (measured)  size  of  delivered  code.  O 

•  Staffing  profile.  Q 

•  Labor  mix.  □ 

•  Skill  level  of  the  project  team,  measured  relative  to  the  skill  Q 

level  of  the  organization's  typical  team. 

•  Nonlabor  costs.  □ 

•  Management  costs.  □ 

•  System  integration  costs.  Q 

•  An  estimate  at  completion.  □ 


(What  would  have  been  estimated  at  the  start,  had  we 
known  then  what  we  know  now,  together  with  a  record  of 
the  values  and  rationales  used  to  map  the  cost  model 
parameters  to  actual  organization  performance.) 


•  Extenuating  circumstances  or  reasons  for  the  differences  O 

between  the  original  and  final  estimates. 

•  A  work  breakdown  structure  or  alternative  description  of  □ 

tasks  included  in  the  recorded  costs. 

•  A  work-flow  schematic  for  the  software  process.  Q| 

(So  that  differences  in  processes  are  made  visible  and 
effects  of  process  improvements  can  be  tracked.) 

•  A  summary  or  list  of  significant  deliverables  produced  by  the  □ 
project  (software,  documentation,  etc.). 

•  A  summary  of  any  unusual  issues  or  contract  factors  that  Q 

affected  cost  or  schedule. 

•  If  multiple  builds  or  releases  are  used,  the  size,  cost,  Q 

schedule,  and  characteristics  of  each  build  or  release. 
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Requisite  2.  Structured  processes  for  estimating  product 
size  and  reuse 

Evidence  of  Maturity 

The  estimating  processes  for  size  and  reuse  are  documented. 

The  estimating  processes  for  size  and  reuse  are  followed. 

The  descriptions  of  size  and  reuse  identify  what  has  been 
included  in  (and  excluded  from)  the  size  and  reuse  measures. 

The  measures  of  reuse  distinguish  between  code  that  will  be 
modified  and  code  that  will  be  integrated  as-is  into  the  system. 

Size  estimates  are  checked  by  relating  them  to  measured  sizes 
of  other  software  products  or  components. 

The  size  estimating  process  is  checked  periodically  by 
comparing  its  predictive  capabilities  with  measured  sizes  of 
completed  products. 

Because  size  estimating  is  often  the  weakest  link  in  cost  and 
schedule  estimating,  the  organization  has  a  continuing  effort  that 
focuses  on  improving  its  size  estimating  process. 


□ 

□ 

□ 

□ 

□ 

□ 

□ 
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Requisite  3.  Mechanisms  for  extrapolating  from  demon¬ 
strated  accomplishments  on  past  projects 

Evidence  of  Maturity 

The  extrapolation  process  is  documented. 

Cost  models  and  other  tools  have  been  acquired  (or  developed) 
to  assist  estimators. 

The  cost  models  have  been  calibrated  to  relevant  historical  data 

Cost  model  calibrations  are  up  to  date. 

The  cost  and  schedule  models  are  used  to  quantify  demon¬ 
strated  organizational  performance  in  ways  that  normalize  for 
differences  among  software  products  and  projects. 

The  consistency  that  estimators  achieve  when  fitting  cost  models 
to  historical  data  is  measured  and  tracked. 

Values  used  for  cost  model  parameters  are  validated  by 
comparisons  with  past  projects. 

The  methods  used  to  account  for  reuse  recognize  that  reuse  is 
not  free. 

(Estimates  account  for  activities  such  as  interface  design, 
modification,  integration,  testing,  and  documentation  that 
are  associated  with  effective  reuse.) 

Extrapolations  from  past  projects  incorporate  measured  trends 
in  technology  improvement,  either  within  the  cost  models 
themselves  or  as  inputs  to  them. 

Estimators  work  jointly  with  project  managers  and  experienced 
technical  people  to  identify  how  the  new  work  compares  to  work 
the  organization  or  others  have  done  before. 

More  than  one  cost  model  or  estimating  approach  is  used,  and 
differences  among  results  are  analyzed  and  explained. 

Trends  in  the  organization's  process  and  performance 
parameters  are  tracked  to  identify  their  effects  on  cost  model 
calibrations. 


Requisite  4.  Audit  trails  (The  values  for  the  cost  model 
parameters  used  to  produce  each  estimate 
are  recorded  and  explained.) 

Evidence  of  Maturity 

The  organization's  process  documentation  identifies  who  is 
responsible  for  preparing  the  audit  trail  for  software  estimates. 

A  list  of  parameter  values  and  their  rationales  accompanies 
each  estimate. 

A  template  or  format  is  used  to  record  the  values  of  cost  model 
parameters  and  their  rationales. 

(This  helps  avoid  oversights.) 

Uncertainties  in  parameter  values  are  identified  and  quantified. 
(For  use  in  risk  analyses.) 

The  lists  of  parameter  values  and  their  rationales  are  retained  in 
the  organization's  historical  database. 


Requisite  5.  Integrity  in  dealing  with  dictated  costs  and 

schedules  (Imposed  answers  are  acceptable 
only  when  legitimate  design-to-cost  or  plan- 
to-cost  processes  are  followed.) 

Evidence  of  Maturity 

Management  reviews  and  agrees  to  parameter  values  and 
rationales  before  costs  are  estimated. 

Reasons  for  changing  parameter  values  from  those  identified  in 
the  calibration  set  are  documented. 

Adjustments  to  cost  model  parameters  to  meet  desired  costs  or 
schedules  are  accompanied  by  management  actions  that  make 
the  parameter  values  realistic. 

The  actions  that  the  organization  intends  to  take  to  make  its 
adjusted  cost  model  parameters  valid  are  spelled  out  in  the 
project  plan. 


Requisite  6.  Data  collection  and  feedback  processes  that 
foster  capturing  and  correctly  interpreting 
data  from  work  performed 

Evidence  of  Maturity 

There  is  a  defined  process  for  gathering  information  on 
completed  projects  and  entering  it  into  the  historical  database. 

□ 

Postmortems  are  held  at  the  completion  of  each  project. 

•  To  ensure  that  recorded  data  are  valid. 

□ 

•  To  ensure  that  events  that  affected  cost  or  schedule  get 
recorded  and  described  while  they  are  still  fresh  in  people's 
minds. 

Estimates  used  for  original  project  planning  are  saved  and 
entered  into  the  historical  database. 

□ 

Re-estimates  and  estimates  for  changes  to  the  product  or 
process  are  recorded  and  saved  in  the  historical  database. 

□ 

Pilots  and  prototypes  of  new  software  processes  are  measured 
and  tracked  to  capture  information  that  can  guide  estimates  for 
full-scale  processes. 

□ 

Organizations  that  acquire  software  receive  and  save  copies  of 
the  developer's  postmortem  reports. 

□ 

There  is  a  structured  process  for  capturing  data  on  effort  and 
cost  from  ongoing  and  completed  projects. 

□ 

The  capturing  of  data  for  cost  estimating  and  planning  is 
integrated  with  the  measurement  processes  used  for  project 
tracking  and  oversight  and  process  improvement. 

□ 

Estimates-to-complete  are  updated  and  reviewed  at  regularly 
scheduled  intervals  (e.g.,  monthly). 

□ 

Estimates-to-complete  are  updated  and  reviewed  whenever 
there  is  a  major  change  to  requirements,  resources,  priorities, 
commitments,  assumptions,  or  understanding  of  the  project. 

□ 

The  processes  for  capturing,  collecting,  and  disseminating 
measurement  results  and  descriptive  data  are  supported  by 
automation,  so  that  opportunities  for  misinformation,  sloppiness, 
and  indifference  are  minimized. 

□ 

Page  6  of  6 


Indicators  of  Estimating  Capability 
—  A  Checklist  for  Successful  Estimating  Environments  — 

Successful  estimating  processes  do  not  appear  spontaneously. 
Building  and  sustaining  a  software  estimating  capability  requires 
organizational  commitment  and  action.  This  checklist  is  designed  to 
help  you  assess  the  quality  of  the  support  that  an  organization  provides 
for  its  estimating  process.  It  identifies  elements  of  evidence  to  consider 
when  evaluating  the  infrastructure  that  supports  that  process. 


Indicator  1.  Management  acknowledges  its  responsibility 
for  developing  and  sustaining  an  estimating 
capability. 

Evidence  of  Maturity 

Estimating  is  treated  as  a  corporate  process  that  is  essential  to  Q 

the  organization's  business  success. 

The  individual  or  office  responsible  for  establishing  and  Q 

sustaining  the  organization's  estimating  capability  has  been 
clearly  identified. 

At  least  one  person  in  the  organization  has  a  standing  Q 

assignment  or  responsibility  as  an  estimator. 

(Continuity  and  experience  are  needed  for  sustaining  and 
improving  the  organization's  corporate  memory  and 
estimating  capabilities.) 

Two  or  more  people  have  standing  responsibilities  or  Q 

assignments  as  estimators. 

(To  provide  backup  capability  and  reinforcement  of 
professional  skills.) 

Estimating  assignments  and  responsibilities  have  been  in  place  Q 
long  enough  for  the  organization  to  develop  competence  in 
software  estimating. 


Indicator  2.  The  estimating  function  is  supported  by  a 
budget  and  funds. 

Evidence  of  Maturity 

The  estimating  function  is  a  line  item  in  the  organization's  Q 

budget  and  staffing  plans. 

People  and  funds  have  been  allocated  to  support  the  estimating  Q 
function. 
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The  budget  and  funds  provide  for 

•  Establishing  and  sustaining  a  corporate  memory.  Q 

•  Preparing  estimates.  Q 

•  Capturing  data  from  ongoing  and  completed  projects.  Q 

•  Acquiring  and  using  estimating  models  and  tools.  Q 

•  Educating  people  to  be  estimators.  Q 

•  Training  in  the  use  of  cost  models  and  tools.  Q 

Budgets  and  funds  in  previous  years  were  sufficient  to  develop  a  O 
current  estimating  capability. 

The  current  budget  and  funds  are  adequate  for  sustaining  and  Q 

improving  the  estimating  capability. 


Indicator  3.  Estimators  have  been  equipped  with  the  tools 
and  training  needed  for  reliable  estimating. 

Evidence  of  Maturity 

Estimators  have  up-to-date  desktop  computing  facilities 
(hardware  and  software). 

Cost  models  and  other  software  such  as  spreadsheets, 
databases,  and  statistical  programs  have  been  acquired  or 
developed. 

Estimators  have  received  training  in  the  cost,  schedule,  and  size 
models  they  use. 

Estimators  have  received  training  in  estimating. 


□ 

□ 

□ 

□ 


Indicator  4.  The  people  assigned  as  estimators  are 
experienced  and  capable. 

Evidence  of  Maturity 

The  estimators  have  professional  experience  with  the  processes 
and  products  whose  costs,  schedules,  or  sizes  they  estimate. 

Estimators  have  educational  backgrounds  that  support 
quantitative  analysis. 


The  number  of  years  of  estimating  experience  among  people  r~j 

assigned  as  estimators  is  computed  and  tracked. 

The  estimators  participate  in  professional  activities  and  societies  Q 
related  to  estimating. 
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Indicator  5.  Recognition  and  career  paths  exist  such  that 
qualified  people  want  to  serve  as  estimators. 

Evidence  of  Maturity 

Estimating  is  viewed  by  both  employees  and  managers  as  a 
career-broadening  assignment. 

Previous  estimators  have  moved  on  to  positions  of  equal  or 
higher  responsibility. 

People  ask  to  become  estimators. 


Indicator  6.  Estimators  work  with  process  improvement 
teams  to  quantify  and  track  progress  in 
software  process  improvement. 

Evidence  of  Maturity 

Estimators  use  their  cost  models  to  account  for  factors  that  make 
projects  different,  so  that  effects  of  process  improvements  can  be 
meaningfully  measured  and  compared. 

The  organization  uses  trend  analyses  derived  from  cost-model 
calibrations  to  track  progress  in  its  software  process 
performance. 


Indicator  7.  The  estimating  capability  of  the  organization 
is  quantified,  tracked,  and  evaluated. 

Evidence  of  Maturity 

Management  tracks  and  reviews  the  effectiveness  of  its 
estimating  processes. 

Managers  and  other  users  of  estimates  are  interviewed 
periodically  to  identify 

•  The  estimating  needs  that  are  being  met. 

•  The  estimating  needs  that  are  not  being  met. 

•  Opportunities  for  improving  the  estimating  process. 


Elements  of  Good  Estimating  Practice 

This  is  a  list  of  practices  that  can  help  organizations  produce  reliable 
cost  and  schedule  estimates.  It  includes  examples  that  do  not  fit  neatly 
in  a  checklist  of  process  requisites,  but  that  are  worth  considering  when 
implementing  a  software  estimating  process  or  assessing  its  maturity. 

Element  Evidence  of  Maturity 

1.  The  purpose  and  objec-  The  purpose  and  objectives  of  each  estimate 

tives  of  each  estimate  are  are  stated  in  writing. 

clearly  understood  by  both  (Differing  purposes  or  objectives  affect  the 

the  producer  and  the  user  way  an  estimate  should  be  interpreted  or 

of  the  estimate.  used.  Examples  include  feasibility 

studies,  planning  estimates,  rough  order- 
of-magnitude  estimates,  formal  estimates, 
proposal  estimates,  bid  evaluation 
estimates,  evaluations  of  alternative 
designs  or  management  strategies, 
estimates  to  complete,  estimates  at 
completion,  life-cycle  cost  estimates,  risk 
analyses,  and  design-to-cost  studies.) 


Estimates  for  product  size  and  content  are 
backed  up  by  systematic  engineering 
analyses. 

The  terms  and  parameters  that  describe  the 
product  permit  comparisons  to  be  made  with 
other  products. 

(The  parameter  sets  of  cost  models 
provide  useful  frameworks  for  this 
purpose.) 


3.  Tasks  to  be  estimated  are  Estimators  use  checklists  to  identify  the 
clearly  identified.  elements  and  activities  that  are  included  in 

(and  excluded  from)  estimates. 

Proposal  teams  use  checklists  to  ensure  that 
the  proposal  (and  the  estimate)  covers  all 
aspects  of  the  work  to  be  performed. 

Mappings  to  the  contract  work  breakdown 
structure  are  documented. 
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2.  The  products  to  be 
produced  are  clearly 
described. 


4.  People  from  related  but 
different  projects  or 
disciplines  are  involved  in 
the  estimating  process. 


Estimators  are  included  in  proposal  kickoff 
meetings. 

Estimators  work  closely  with  project  managers 
and  the  technical  staff  from  the  start  of  the 
project,  as  part  of  the  proposal  team. 

If  integrated  project  teams  are  used,  they 
include  at  least  one  estimator. 


5.  Estimates  are  validated  by 
relating  them  to 
demonstrated  performance 
on  completed  projects. 


Cost  model  calibrations  are  used  to  develop 
organizational  proficiency  and  consistency  in 
the  ways  the  organization  relates  its 
parameter  values  to  descriptions  of  projects. 

Values  assigned  to  cost  model  parameters  are 
based  on  comparisons  with  values  that  give 
good  fits  to  completed  projects. 

Reasons  are  documented  for  the  values 
assigned  to  each  cost  model  parameter. 

Calibrations  are  performed  and  estimates  are 
reviewed  (or  prepared)  by  an  estimator  who 
has  organizational  perspectives  and 
experience,  so  that  they  draw  on  the  full 
experience  of  the  organization,  not  just  the 
views  of  the  project. 


6.  More  than  one  cost  model 
or  estimating  approach  is 
used. 


Differences  in  results  are  analyzed  and 
accounted  for. 

Records  are  kept  of  the  effectiveness  of  the 
different  models  for  different  applications  or 
life-cycle  phases,  so  that  effective 
combinations  of  estimates  from  different 
models  can  be  used  for  future  estimates. 


7.  Potential  cost  and  schedule  A  structured  process  is  used  to  identify  and 
impacts  are  estimated  for  scope  technical  risks. 

all  identified  risks.  , ,  ... 

Uncertainties  in  values  of  cost  model 

parameters  are  identified  and  quantified. 

The  effects  of  uncertainties  in  descriptive 
parameters  are  evaluated  and  reported  along 
with  estimates. 
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8.  Dictated  schedules  (if  Managers  (and  customers,  where  appropriate) 

present)  are  analyzed  for  are  informed  of  potential  cost  savings 
impacts  on  cost.  associated  with  alternative  schedules. 


9.  Dictated  costs  and 

schedules  (if  present)  are 
analyzed  for  feasibility. 
The  analyses  identify  the 
management  alternatives 
and  changes  that  must  be 
made  to  key  cost  drivers  if 
the  targets  are  to  be 
achievable. 


Managers  (and  customers,  where  appropriate) 
are  informed  of  potential  ways  to  meet  target 
costs  and  schedules. 

Managers  (and  customers,  where  appropriate) 
are  informed  of  changes  that  must  be  made  to 
key  cost  drivers  if  target  costs  and  schedules 
are  to  be  achievable. 

Estimators  are  not  forced  to  give  unrealistic 
cost  or  schedule  estimates. 


10.  Estimates  are  kept  current.  Estimates  (or  estimates-to-complete)  are 

updated  whenever 

•  Changes  to  requirements  affect  cost  or 
schedule. 

•  Constraints  change. 

•  Resources  change. 

•  Priorities  change. 

•  Actual  values  for  product  or  process 
parameters  are  found  to  be  significantly 
different  from  those  on  which  the  plan  is 
based. 

•  Tracking  measures  indicate  that  critical 
path  tasks  cannot  be  completed  as 
planned. 


1 1 .  The  results  of  estimates  are  Plans  are  reviewed  and  updated  whenever 

integrated  with  project  estimates  change. 

planning  and  tracking.  _  , ,  .... 

The  estimates  used  for  project  planning  are 

also  used  as  baselines  for  project  tracking. 

Feedback  from  project  tracking  is  used  to 
improve  both  the  estimating  and  development 
processes. 
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12.  The  organization  has  a 
historical  database  for 
organizing  and  retaining 
information  on  completed 
projects. 


The  historical  database  is  treated  as  an 
integral  part  of  the  estimating  process,  and 
estimators  have  strong  and  active  roles  in 
specifying  and  sustaining  the  information  it 
contains. 

The  elements  included  in  (and  excluded  from) 
effort,  cost,  schedule,  size,  and  reuse 
measures  are  clearly  identified. 

(For  examples,  see  the  SEI  checklists  for 
effort,  schedule,  and  size  measurement.) 

Schedule  milestones  (start  and  finish  dates) 
are  described  by  criteria  for  initiation  or 
completion. 

(So  that  work  accomplished  between 
milestones  is  clearly  bounded.) 

Effort  and  cost  data  clearly  indicate  which 
parts  of  the  life  cycle  and  which  activities  are 
covered  by  the  different  categories  of  effort  or 
costs  recorded. 

Records  for  projects  indicate  whether  or  not 
unpaid  overtime  has  been  used. 

The  amount  of  unpaid  overtime  is  quantified. 
(Measures  or  estimates  of  unpaid 
overtime  for  each  project  are  made  and 
recorded,  so  that  historical  costs  can 
provide  a  valid  basis  for  estimating  future 
effort.) 

The  database  contains  a  useful  set  of 
completed  projects. 

Cost  models  are  used  to  provide  consistent 
frameworks  (standard  terms  and  parameters) 
for  recording  historical  data. 

Historical  data  are  examined  to  identify 
inconsistencies,  and  anomalies  are  corrected 
or  explained. 

(This  is  perhaps  best  done  with  the  same 
cost  models  that  are  used  for  estimating.) 
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13.  Information  on  completed 
projects  is  captured  and 
entered  into  the  historical 
database. 


Original  estimates  for  size,  cost,  and  schedule 
are  retained. 

Postmortems  are  held  at  the  completion  of 
each  project. 

•  To  ensure  that  recorded  data  are  valid. 

•  To  ensure  that  events  that  have  affected 
costs  or  schedules  are  described  and 
recorded  while  they  are  still  fresh  in 
people's  minds. 


14.  The  emphasis  throughout 
is  on  developing 
consistency  in  describing 
completed  projects  and  in 
relating  new  work  to 
demonstrated  performance 
on  those  projects. 


The  consistency  achieved  when  calibrating 
cost  models  to  completed  projects  is 
measured  and  tracked. 

The  phrase  "model  accuracy"  (implying  that 
the  model  rather  than  the  estimator  made  the 
estimate)  is  never  used. 
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